NetEx: Cost-effective Bulk Data Transfers for Cloud Computing

Cloud computing offers individuals and organizations the potential for reducing their IT costs. In this paper, we focus on the problem of high bandwidth prices charged by cloud providers for customers’ data uploads and downloads. The cost of such data transfers can become prohibitive when the volume of data transferred is large. The high price of data transfers reflect the cost of raw bandwidth that cloud providers pay to transit ISPs. Raw bandwidth is expensive because ISPs need to overprovision their networks for peak utilization. In this paper, we propose that ISPs use the spare capacity on their backbone links to deliver bulk data. Since ISPs make more effective utilization of otherwise unused bandwidth, they can offer this service at lower prices, which will benefit cloud providers and cloud users. Cloud users could use this service to ship delay-tolerant data, e.g., data backups, software distributions, and large data sets. We present NetEx, a bulk transfer system that opportunistically exploits the excess capacities of network links to deliver bulk content cheaply and efficiently. NetEx uses bandwidthaware routing, which adapts to dynamically changing available bandwidth across potentially multiple paths between the source and the destination of a bulk transfer. Because NetEx works in the background scavenging unused bandwidth, ISPs can easily deploy it over routers that support simple priority queueing without affecting existing Internet traffic or routing. We evaluated NetEx using data gathered from the backbone of a large commercial Tier-1 ISP. Our results show that NetEx achieves near-optimal utilization of spare link bandwidth and that ISPs can use it to deliver 60% to 170% more data than what they transfer today.